-
-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix error on instance property and init-only variable with the same name in a dataclass #17219
base: master
Are you sure you want to change the base?
Fix error on instance property and init-only variable with the same name in a dataclass #17219
Conversation
…ame in a dataclass
This comment has been minimized.
This comment has been minimized.
The actual fix for the reported bug looks like it works to me, and I think it's probably a good thing for But I'm a little concerned that the example reported in the bug is actually relying on undefined behavior that just sort of happens to work.
does indeed work at runtime, because dataclasses relies on But the dataclasses docs do not attempt to specify behavior here, nor do the unit tests at https://github.com/python/cpython/blob/main/Lib/test/test_dataclasses/__init__.py validate anything. And we can very easily run into poorly-defined behavior; for example if I tweak the first two lines in the example valid to provide a default argument but this code
then it will not work as expected, trying to use the default value will give
because even though dataclasses is interpreting the annotation using cc @JelleZijlstra / @hauntsaninja - I think this question is close to being either a typing spec or a CPython standard library question. |
if name in cls.names: | ||
node = cls.names[name].node | ||
if isinstance(node, FuncBase): | ||
return node | ||
elif isinstance(node, Decorator): # Two `if`s make `mypyc` happy | ||
return node | ||
else: | ||
return None | ||
elif possible_redefinitions := sorted( | ||
[n for n in cls.names.keys() if n.startswith(f"{name}-redefinition")] | ||
): | ||
node = cls.names[possible_redefinitions[-1]].node |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regardless of what we decide to do about dataclasses that redefine InitVar
fields, this seems like it might be a good change.
In general, validating that there aren't redefinitions (which is happening in semanal
) seems like it should be independent of getting the appropriate value for a name; when dealing with dependencies, using the last-defined value for a name seems like a good thing.
I wonder if it would be worth pulling into a helper function, and using not just for methods but other attributes as well?
Oh, nice catch! I didn't fully realise that InitVar could have a default. From the CPython side, redefinition in general is problematic, even with normal fields. In my similar-to-dataclasses library, I have some heuristic based runtime checks against this case. Basically I complain if something in
(my similar-to-dataclasses lib is sometimes used in ways that make it more likely that people will try to clobber things) CPython is probably stuck with its behaviour given that folks are clearly relying on it. Either way, it should definitely have tests for this case. Want to open a PR? From the mypy side, it seems like a half dozen folks have run into this, so it would be nice to support. The ideal behaviour would match the runtime behaviour, i.e. we allow redefinition when there isn't a value and we disallow redefinition when there is a value. @roberfi are you interested in adding the warning back when there's a default value? :-) |
Oh, that's crazy! This kind of implementation could never have a default value. Thank you so much for the catch and the explanation. It was a very good one!
Yes, sure, I will add the check/warning and a test to verify that it's working. Let's do it! |
Yeah, I can look into adding some CPython unit tests later this week |
This comment has been minimized.
This comment has been minimized.
As originally discussed in python/mypy#17219, MyPy has had a false-positive bug report because it errors when a dataclass has methods that shadow an `InitVar` field. It is actually a bit surprising that this works, it turns out that `__annotations__` "remembers" field assignments even if the bound names are later overwritten by methods; it will *not* work to provide a default value. There have been multiple bug reports on MyPy so we know people are actually relying on this in practice; most likely it comes up when a dataclass wants to take a "raw" value as an InitVar and transform it somehow in `__post_init__` into a different value before assigning it to a field; in that case they may choose to make the actual field private and provide a property for access. I currently provide a test of the happy path where there is no default value provided, but no tests of the behavior when no default is provided (in which case the property will override the default) and no documentation (because I'm not sure we want to consider this behavior officially supported). The main goal is to have a regression test since it would be easy for a refactor to break this.
As originally discussed in python/mypy#17219, MyPy has had a false-positive bug report because it errors when a dataclass has methods that shadow an `InitVar` field. It is actually a bit surprising that this works, it turns out that `__annotations__` "remembers" field assignments even if the bound names are later overwritten by methods; it will *not* work to provide a default value. There have been multiple bug reports on MyPy so we know people are actually relying on this in practice; most likely it comes up when a dataclass wants to take a "raw" value as an InitVar and transform it somehow in `__post_init__` into a different value before assigning it to a field; in that case they may choose to make the actual field private and provide a property for access. I currently provide a test of the happy path where there is no default value provided, but no tests of the behavior when no default is provided (in which case the property will override the default) and no documentation (because I'm not sure we want to consider this behavior officially supported). The main goal is to have a regression test since it would be easy for a refactor to break this.
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
Fixes #12046
It does not raise "already defined" fail if already existing node is
InitVar
, so name of the property will be redefined. Then, when getting the method of that property, it looks for a possible redefinition.